Applications of Weighted Automata in Natural Language Processing

Linguistics and automata theory were at one time tightly knit. Very early on, finite-state processes were used by Markov [35, 27] to predict sequences of vowels and consonants in novels by Pushkin. Shannon [48] extended this idea to predict letter sequences of English words using Markov processes. While many theorems about finite-state acceptors (FSAs) and finite-state transducers (FSTs) were proven in the 1950s, Chomsky argued that such devices were too simple to adequately describe natural language [6]. Chomsky employed context-free grammars (CFGs) and then introduced the more powerful transformational grammars (TG), loosely defined in [7]. In attempting to formalize TG, automata theorists like Rounds [46] and Thatcher [52] introduced the theory of tree transducers. Computational linguistics also got going in earnest, with Woods’ use of augmented transition networks (ATNs) for automatic natural language parsing. In the final paragraph of his 1973 tree automata survey [53], Thatcher wrote: